Content Analysis of Compressed Video

Continuous video applications such as video transfer, searching and display can save time and space by operating on compressed data. The problem is that most mid and high level image operations, such as object recognition, are formulated as sequences of operations in the image domain. Such methods need direct access to pixel information as a starting point, but the pixel information in a compressed image stream is not immediately accessible. In this paper we show how to perform object recognition directly on compressed images or continuous video streams without recovering explicit pixel information. Our approach is based on eigenvectors constructed from compressed image data. Distance in eigenspace, which is the basis for classiication, is preserved under linear orthogonal transformations. The Discrete Cosine Transform therefore preserves distance in eigenspace, allowing for exactly the same classiication results on transformed images. We show that the classiication measure is robust under high levels of subsequent quantization. The MPEG and JPEG compression standards are based on the Discrete Cosine Transform followed by quantization. We use MPEG and JPEG data in our experiments, which show that classiication using lossy-compressed images performs as well as the corresponding method in the image domain. Our performance results show that a ve-fold speedup can be gained by using compressed data.